ReGO: Reference-Guided Outpainting for Scenery Image.

We present ReGO (Reference-Guided Outpainting), a new method for the task of sketch-guided image outpainting. Despite the significant progress made in producing semantically coherent content, existing outpainting methods often fail to deliver visually appealing results due to blurry textures and generative artifacts. To address these issues, ReGO leverages neighboring reference images to synthesize texture-rich results by transferring pixels from them. Specifically, an Adaptive Content Selection (ACS) module is incorporated into ReGO to facilitate pixel transfer for texture compensating of the target image. Additionally, a style ranking loss is introduced to maintain consistency in terms of style while preventing the generated part from being influenced by the reference images. ReGO is a model-agnostic learning paradigm for outpainting tasks. In our experiments, we integrate ReGO with three state-of-the-art outpainting models to evaluate its effectiveness. The results obtained on three scenery benchmarks, i.e. NS6K, NS8K and SUN Attribute, demonstrate the superior performance of ReGO compared to prior art in terms of texture richness and authenticity. Our code is available at https://github.com/wangyxxjtu/ReGO-Pytorch.

Exploration of radionuclide labeling of a novel scFv-Fc fusion protein targeting CLDN18.2 for tumor diagnosis and treatment.

PURPOSE: Claudin 18.2 (CLDN18.2), due to its highly selective expression in tumor cells, has made breakthrough progress in clinical research and is expected to be integrated into routine tumor diagnosis and treatment. METHODS: In this research, we obtained an scFv-Fc fusion protein (SF106) targeting CLDN18.2 through hybridoma technology. The scFv-Fc fusion protein was labeled with radioactive isotopes (124I and 177Lu) to generate the radio-probes. The targeting and specificity of the radio-probes were tested in cellular models, and its diagnostic and therapeutic potential was further evaluated in tumor-bearing models. RESULTS: The molecular probes [124I]I-SF106 and [177Lu]Lu-DOTA-SF106 possess high radiochemical purity (RCP, 98.18 ± 0.93 % and 97.05 ± 1.1 %) and exhibit good stability in phosphate buffer saline and 5 % human serum albumin (92.44 ± 4.68 % and 91.03 ± 2.42 % at 120 h). [124I]I-SF106 uptake in cells expressing CLDN18.2 was well targeted and specific, and the dissociation constant was 17.74 nM [124I]I-SF106 micro-PET imaging showed that the maximum standardized uptake value (SUVmax) was significantly higher than CLDN18.2-negative tumors (1.83 ± 0.02 vs. 1.23 ± 0.04, p < 0.001). The maximum uptake was attained in tumors expressing CLDN18.2 at 48 h after injection. [124I]I-SF106 and [177Lu]Lu-DOTA-SF106 dosimetric study showed that the effective dose in humans complies with the medical safety standards required for their clinical application. The results of treatment experiments showed that 3 MBq of [177Lu]Lu-DOTA-SF106 in CLDN18.2-expressing tumor-bearing mice could significantly inhibit tumor growth. CONCLUSION: These results indicate that radionuclide-labeled scFv-Fc molecular probes ([124I]I-SF106 and [177Lu]Lu-DOTA-SF106) provide a new possibility for the diagnosis and treatment of CLDN18.2-positive cancer patients in clinical practice.

[177Lu]Lu-labeled anti-claudin-18.2 antibody demonstrated radioimmunotherapy potential in gastric cancer mouse xenograft models.

PURPOSE: Gastric cancer (GC), one of the most prevalent and deadliest tumors worldwide, is often diagnosed at an advanced stage with limited treatment options and poor prognosis. The development of a CLDN18.2-targeted radioimmunotherapy probe is a potential treatment option for GC. METHODS: The CLDN18.2 antibody TST001 (provided by Transcenta) was conjugated with DOTA and radiolabeled with the radioactive nuclide 177Lu. The specificity and targeting ability were evaluated by cell uptake, imaging and biodistribution experiments. In BGC823CLDN18.2/AGSCLDN18.2 mouse models, the efficacy of [177Lu]Lu-TST001 against CLDN18.2-expressing tumors was demonstrated, and toxicity was evaluated by H&E staining and blood sample testing. RESULTS: [177Lu]Lu-TST001 was labeled with an 99.17%±0.32 radiochemical purity, an 18.50 ± 1.27 MBq/nmol specific activity and a stability of ≥ 94% after 7 days. It exhibited specific and high tumor uptake in CLDN18.2-positive xenografts of GC mouse models. Survival studies in BGC823CLDN18.2 and AGSCLDN18.2 tumor-bearing mouse models indicated that a low dose of 5.55 MBq and a high dose of 11.10 MBq [177Lu]Lu-TST001 significantly inhibited tumor growth compared to the saline control group, with the 11.1 MBq group showing better therapeutic efficacy. Histological staining with hematoxylin and eosin (H&E) and Ki67 immunohistochemistry of residual tissues confirmed tumor tissue destruction and reduced tumor cell proliferation following treatment. H&E showed that there was no significant short-term toxicity observed in the heart, spleen, stomach or other important organs when treated with a high dose of [177Lu]Lu-TST001, and no apparent hematotoxicity or liver toxicity was observed. CONCLUSION: In preclinical studies, [177Lu]Lu-TST001 demonstrated significant antitumor efficacy with acceptable toxicity. It exhibits strong potential for clinical translation, providing a new promising treatment option for CLDN18.2-overexpressing tumors, including GC.

Forecasting Treatment Outcomes Over Time Using Alternating Deep Sequential Models.

Medical decision making often relies on accurately forecasting future patient trajectories. Conventional approaches for patient progression modeling often do not explicitly model treatments when predicting patient trajectories and outcomes. In this paper, we propose Alternating Transformer (AL-Transformer) to jointly model treatment dynamics and clinical outcomes over time as alternating sequential models. To predict the sparse treatment, a constraint learned by a CNN is used to constrain the sparse treatment output. Additionally, we leverage causal convolution in the self-attention mechanism of AL-Transformer to incorporate local spatial information in the sequence, thus enhancing the model's ability to capture local contextual information of the sequence. Experimental results on two datasets from patients with sepsis and respiratory failure extracted from the Medical Information Mart for Intensive Care (MIMIC) database demonstrate the effectiveness of the proposed approach, outperforming existing state-of-the-art methods.We will release the code on Github after the paper is accepted.

4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement.

Image enhancement aims at improving the aesthetic visual quality of photos by retouching the color and tone, and is an essential technology for professional digital photography. Recent years deep learning-based image enhancement algorithms have achieved promising performance and attracted increasing popularity. However, typical efforts attempt to construct a uniform enhancer for all pixels' color transformation. It ignores the pixel differences between different content (e.g., sky, ocean, etc.) that are significant for photographs, causing unsatisfactory results. In this paper, we propose a novel learnable context-aware 4-dimensional lookup table (4D LUT), which achieves content-dependent enhancement of different contents in each image via adaptively learning of photo context. In particular, we first introduce a lightweight context encoder and a parameter encoder to learn a context map for the pixel-level category and a group of image-adaptive coefficients, respectively. Then, the context-aware 4D LUT is generated by integrating multiple basis 4D LUTs via the coefficients. Finally, the enhanced image can be obtained by feeding the source image and context map into fused context-aware 4D LUT via quadrilinear interpolation. Compared with traditional 3D LUT, i.e., RGB mapping to RGB, which is usually used in camera imaging pipeline systems or tools, 4D LUT, i.e., RGBC(RGB+Context) mapping to RGB, enables finer control of color transformations for pixels with different content in each image, even though they have the same RGB values. Experimental results demonstrate that our method outperforms other state-of-the-art methods in widely-used benchmarks.

TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation.

Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive frames. State-of-the-art approaches usually adopt a two-step solution, which includes 1) generating locally-warped pixels by calculating the optical flow based on pre-defined motion patterns (e.g., uniform motion, symmetric motion), 2) blending the warped pixels to form a full frame through deep neural synthesis networks. However, for various complicated motions (e.g., non-uniform motion, turn around), such improper assumptions about pre-defined motion patterns introduce the inconsistent warping from the two consecutive frames. This leads to the warped features for new frames are usually not aligned, yielding distortion and blur, especially when large and complex motions occur. To solve this issue, in this paper we propose a novel Trajectory-aware Transformer for Video Frame Interpolation (TTVFI). In particular, we formulate the warped features with inconsistent motions as query tokens, and formulate relevant regions in a motion trajectory from two original consecutive frames into keys and values. Self-attention is learned on relevant tokens along the trajectory to blend the pristine features into intermediate frames through end-to-end training. Experimental results demonstrate that our method outperforms other state-of-the-art methods in four widely-used VFI benchmarks. Both code and pre-trained models will be released at https://github.com/ChengxuLiu/TTVFI.

Development of a CLDN18.2-targeting immuno-PET probe for non-invasive imaging in gastrointestinal tumors.

Claudin18.2 (CLDN18.2) is a tight junction protein that is overexpressed in a variety of solid tumors such as gastrointestinal cancer and oesophageal cancer. It has been identified as a promising target and a potential biomarker to diagnose tumor, evaluate efficacy, and determine patient prognosis. TST001 is a recombinant humanized CLDN18.2 antibody that selectively binds to the extracellular loop of human Claudin18.2. In this study, we constructed a solid target radionuclide zirconium-89 (89Zr) labled-TST001 to detect the expression of in the human stomach cancer BGC823CLDN18.2 cell lines. The [89Zr]Zr-desferrioxamine (DFO)-TST001 showed high radiochemical purity (RCP, >99%) and specific activity (24.15 ± 1.34 GBq/µmol), and was stable in 5% human serum albumin, and phosphate buffer saline (>85% RCP at 96 h). The EC50 values of TST001 and DFO-TST001 were as high as 0.413 ± 0.055 and 0.361 ± 0.058 nM (P > 0.05), respectively. The radiotracer had a significantly higher average standard uptake values in CLDN18.2-positive tumors than in CLDN18.2-negative tumors (1.11 ± 0.02 vs. 0.49 ± 0.03, P = 0.0016) 2 days post injection (p.i.). BGC823CLDN18.2 mice models showed high tumor/muscle ratios 96 h p.i. with [89Zr]Zr-DFO-TST001 was much higher than those of the other imaging groups. Immunohistochemistry results showed that BGC823CLDN18.2 tumors were highly positive (+++) for CLDN18.2, while those in the BGC823 group did not express CLDN18.2 (-). The results of ex vivo biodistribution studies showed that there was a higher distribution in the BGC823CLDN18.2 tumor bearing mice (2.05 ± 0.16 %ID/g) than BGC823 mice (0.69 ± 0.02 %ID/g) and blocking group (0.72 ± 0.02 %ID/g). A dosimetry estimation study showed that the effective dose of [89Zr]Zr-DFO-TST001 was 0.0705 mSv/MBq, which is within the range of acceptable doses for nuclear medicine research. Taken together, these results suggest that Good Manufacturing Practices produced by this immuno-positron emission tomography probe can detect CLDN18.2-overexpressing tumors.

Phase I study of MSB2311, a novel pH-dependent anti-PD-L1 monoclonal antibody, treating patients with advanced solid tumors and lymphoma.

MSB2311 is a novel pH-dependent humanized anti-programmed death-ligand 1 (PD-L1) monoclonal antibody. This phase I study primarily aimed to determine the maximum tolerated dose (MTD)/recommended phase 2 dose level (RP2D) of MSB2311 in patients with advanced solid tumors or lymphoma. MSB2311 was intravenously administered at 3, 10, and 20 mg/kg every 3 weeks (Q3W) and 10 mg/kg every 2 weeks (Q2W) using 3 + 3 design. During expansion phase, eligible patients with either PD-L1 overexpression, Epstein-Barr Virus positive, microsatellite instability high/mismatch repair deficient, or high tumor mutation burden tumors were treated at RP2D. A total of 37 Chinese patients were treated, including 31 with solid tumors and 6 lymphoma. No dose limiting toxicity was reported and MTD was not reached. The trial was expanded at 20 mg/kg Q3W or 10 mg/kg Q2W, both of which were determined as RP2D. Most common drug-related treatment-emergent adverse events were anemia (43.2%), aspartate aminotransferase increase (27.0%), proteinuria (21.6%), alanine aminotransferase increase and hypothyroidism (18.9% each), thyroid stimulating hormone increased and hyperglycemia (16.2% each). Out of 20 efficacy evaluable patients with biomarker positive solid tumors, 6 achieved confirmed partial response with the median duration of response of 11.0 months (95% CI 7.0-11.4) and 4 had stable disease, resulting an objective response rate of 30.0% (95% CI 11.9, 54.3) and disease control rate of 50.0% (95% CI 27.2, 72.8). One partial response was also observed among 6 patients with lymphoma. MSB2311 demonstrated a manageable safety profile and promising antitumor activity in patients with advanced solid tumors and lymphomas.

First-in-human CLDN18.2 functional diagnostic pet imaging of digestive system neoplasms enables whole-body target mapping and lesion detection.

PURPOSE: Claudin 18.2 (CLDN18.2) is a reliable target for lesion detection and could have clinical implications for epithelial tumors, especially digestive system neoplasms. However, there is no predictive technology for accurate whole-body mapping of CLDN18.2 expression in patients. This study assessed the safety of the 124I-18B10(10L) tracer and the feasibility of mapping whole-body CLDN18.2 expression using PET functional imaging. METHODS: The 124I-18B10(10L) probe was synthesized manually, and preclinical experiments including binding affinity and specific targeting ability were conducted after testing in vitro model cells. Patients with pathologically confirmed digestive system neoplasms were enrolled in an ongoing, open-label, single-arm, first-in-human (FiH) phase 0 trial (NCT04883970). 124I-18B10(10L) PET/CT or PET/MR and 18F-FDG PET were performed within one week. RESULTS: 124I-18B10(10L) was successfully constructed with an over 95% radiochemical yield. The results of preclinical experiments showed that it had high stability in saline and high affinity in CLDN18.2 overexpressing cells (Kd = 4.11 nM). Seventeen patients, including 12 with gastric cancers, 4 with pancreatic cancers, and 1 with cholangiocarcinoma were enrolled. 124I-18B10(10L) displayed high uptake in the spleen and liver, and slight uptake in the bone marrow, lung, stomach and pancreas. The tracer uptake SUVmax in tumor lesions ranged from 0.4 to 19.5. Compared with that in lesions that had been treated with CLDN18.2-targeted therapy, 124I-18B10(10L) uptake was significantly higher in lesions that had not. Regional 124I-18B10(10L) PET/MR in two patients showed high tracer uptake in metastatic lymph nodes. CONCLUSIONS: 124I-18B10(10L) was successfully prepared and exhibited a high binding affinity and CLDN18.2 specificity in preclinical studies. As an FiH CLDN18.2 PET tracer, 124I-18B10(10L) was shown to be safe with acceptable dosimetry and to clearly reveal most lesions overexpressing CLDN18.2. TRIAL REGISTRATION: NCT04883970; URL: https://register. CLINICALTRIALS: gov/ . Registered 07 May 2021.

IL-1ß Is an Androgen-Responsive Target in Macrophages for Immunotherapy of Prostate Cancer.

Great attention is paid to the role of androgen receptor (AR) as a central transcriptional factor in driving the growth of prostate cancer (PCa) epithelial cells. However, the understanding of the role of androgen in PCa-infiltrated immune cells and the impact of androgen deprivation therapy (ADT), the first-line treatment for advanced PCa, on the PCa immune microenvironment remains limited. On the other hand, immune checkpoint blockade has revolutionized the treatment of certain cancer types, but fails to achieve any benefit in advanced PCa, due to an immune suppressive environment. In this study, it is reported that AR signaling pathway is evidently activated in tumor-associated macrophages (TAMs) of PCa both in mice and humans. AR acts as a transcriptional repressor for IL1B in TAMs. ADT releases the restraint of AR on IL1B and therefore leads to an excessive expression and secretion of IL-1ß in TAMs. IL-1ß induces myeloid-derived suppressor cells (MDSCs) accumulation that inhibits the activation of cytotoxic T cells, leading to the immune suppressive microenvironment. Critically, anti-IL-1ß antibody coupled with ADT and the immune checkpoint inhibitor anti-PD-1 antibody exerts a stronger anticancer effect on PCa following castration. Together, IL-1ß is an important androgen-responsive immunotherapeutic target for advanced PCa.

PicassoNet: Searching Adaptive Architecture for Efficient Facial Landmark Localization.

Since recent facial landmark localization methods achieve satisfying accuracy, few of them enable fast inference speed, which, however, is critical in many real-world facial applications. Existing methods typically employ complicated network structure and predict all the key points through uniform computation, which is inefficient since individual facial part might take different computation to obtain the best performance. Taking both accuracy and efficiency into consideration, we propose the PicassoNet, a lightweight cascaded facial landmark detector with adaptive computation for individual facial part. Different from the conventional cascaded methods, PicassoNet integrates refinement submodules into a single network with group convolution, where each convolution group predicts landmarks from an individual facial part. Note that the groups' structures are flexible in the training process. Then, a novel grouping search algorithm is proposed to optimize the group division. With formulating the optimization as a network architecture search (NAS) problem, the grouping search adaptively allocates computation to each group and obtains an efficient structure. In addition, we propose a boundary-aware loss to optimize along tangent and normal of facial boundaries, instead of optimizing along horizontal and vertical as the conventional loss (L2, SmoothL1, WingLoss, and so on) do. The novel loss improves the joint locations of predicted keypoints. Experiments on three benchmark datasets AFLW, 300W, and WFLW show that the proposed method runs over 6× times faster than the state of the arts and meanwhile achieves comparable accuracy.

CMID: Crossmodal Image Denoising via Pixel-Wise Deep Reinforcement Learning.

Removing noise from acquired images is a crucial step in various image processing and computer vision tasks. However, the existing methods primarily focus on removing specific noise and ignore the ability to work across modalities, resulting in limited generalization performance. Inspired by the iterative procedure of image processing used by professionals, we propose a pixel-wise crossmodal image-denoising method based on deep reinforcement learning to effectively handle noise across modalities. We proposed a similarity reward to help teach an optimal action sequence to model the step-wise nature of the human processing process explicitly. In addition, We designed an action set capable of handling multiple types of noise to construct the action space, thereby achieving successful crossmodal denoising. Extensive experiments against state-of-the-art methods on publicly available RGB, infrared, and terahertz datasets demonstrate the superiority of our method in crossmodal image denoising.

Diversity-Learning Block: Conquer Feature Homogenization of Multibranch.

Visual Geometry Group (VGG)-style ConvNet is an neural-network process units (NPU)-friendly network; however, the accuracy of this architecture cannot keep up with other well-designed network structures. Although some reparameterization methods are proposed to remedy this weakness, their performance suffers from the homogenization issue of parallel branches, and the preset shape of convolution kernels also influences spatial perception. To address this problem, we propose a diversity-learning (DL) block to build the DLNet, which could adaptively learn various features to enrich the feature space. To balance floating point of operations (FLOPs) and accuracy, groupwise operation is introduced and finally, a lightweight DL ConvNet DLGNet is obtained. Extensive evaluations have been conducted on different computer vision tasks, e.g., image classification Canadian Institute For Advanced Research (CIFAR) and ImageNet, object detection PASCAL visual object classes (VOC) and Microsoft Common Objects in Context (MS COCO), and semantic segmentation (Cityscapes). The experimental results show that our proposed DLGNet can achieve comparable performance with the state-of-the-art networks while the speed is 183% faster than GhostNet and even over 600% faster than MobileNetV3 with similar accuracy when running on NPU.

Product Recognition for Unmanned Vending Machines.

Recently, the emerging concept of "unmanned retail" has drawn more and more attention, and the unmanned retail based on the intelligent unmanned vending machines (UVMs) scene has great market demand. However, existing product recognition methods for intelligent UVMs cannot adapt to large-scale categories and have insufficient accuracy. In this article, we propose a method for large-scale categories product recognition based on intelligent UVMs. It can be divided into two parts: 1) first, we explore the similarities and differences between products through manifold learning, and then we build a hierarchical multigranularity label to constrain the learning of representation; and 2) second, we propose a hierarchical label object detection network, which mainly includes coarse-to-fine refine module (C2FRM) and multiple granularity hierarchical loss (MGHL), which are used to assist in capturing multigranularity features. The highlights of our method are mine potential similarity between large-scale category products and optimization through hierarchical multigranularity labels. Besides, we collected a large-scale product recognition dataset GOODS-85 based on the actual UVMs scenario. Experimental results and analysis demonstrate the effectiveness of the proposed product recognition methods.

Gremlin1 is a therapeutically targetable FGFR1 ligand that regulates lineage plasticity and castration resistance in prostate cancer.

Among the greatest hurdles in clinical management of prostate cancer (PCa) are the progression to lethal castration-resistant prostate cancer (CRPC) and the lack of suitable targeted therapies for advanced disease. Here we identify Gremlin1 as a ligand for fibroblast growth factor receptor 1 (FGFR1), which promotes lineage plasticity and drives castration resistance. Importantly, we generate a specific anti-Gremlin1 therapeutic antibody and demonstrate synergistic effect with androgen deprivation therapy (ADT) in CRPC. GREM1 transcription is suppressed by androgen receptor (AR) and released following ADT. We show that Gremlin1 binds to FGFR1 and activates downstream MAPK signaling. Gremlin1 interacts with FGFR1 differently to its canonical ligand FGF1, as revealed through protein structure docking and mutagenesis experiments. Altogether, our data indicate Gremlin1 as a promising candidate therapeutic target for CRPC.

Domain Adaptive Box-Supervised Instance Segmentation Network for Mitosis Detection.

The number of mitotic cells present in histopathological slides is an important predictor of tumor proliferation in the diagnosis of breast cancer. However, the current approaches can hardly perform precise pixel-level prediction for mitosis datasets with only weak labels (i.e., only provide the centroid location of mitotic cells), and take no account of the large domain gap across histopathological slides from different pathology laboratories. In this work, we propose a Domain adaptive Box-supervised Instance segmentation Network (DBIN) to address the above issues. In DBIN, we propose a high-performance Box-supervised Instance-Aware (BIA) head with the core idea of redesigning three box-supervised mask loss terms. Furthermore, we add a Pseudo-Mask-supervised Semantic (PMS) head for enriching characteristics extracted from underlying feature maps. Besides, we align the pixel-level feature distributions between source and target domains by a Cross-Domain Adaptive Module (CDAM), so as to adapt the detector learned from one lab can work well on unlabeled data from another lab. The proposed method achieves state-of-the-art performance across four mainstream datasets. A series of analysis and experiments show that our proposed BIA and PMS head can accomplish mitosis pixel-wise localization under weak supervision, and we can boost the generalization ability of our model by CDAM.

A contrastive learning approach for ICU false arrhythmia alarm reduction.

The high rate of false arrhythmia alarms in Intensive Care Units (ICUs) can lead to disruption of care, negatively impacting patients' health through noise disturbances, and slow staff response time due to alarm fatigue. Prior false-alarm reduction approaches are often rule-based and require hand-crafted features from physiological waveforms as inputs to machine learning classifiers. Despite considerable prior efforts to address the problem, false alarms are a continuing problem in the ICUs. In this work, we present a deep learning framework to automatically learn feature representations of physiological waveforms using convolutional neural networks (CNNs) to discriminate between true vs. false arrhythmia alarms. We use Contrastive Learning to simultaneously minimize a binary cross entropy classification loss and a proposed similarity loss from pair-wise comparisons of waveform segments over time as a discriminative constraint. Furthermore, we augment our deep models with learned embeddings from a rule-based method to leverage prior domain knowledge for each alarm type. We evaluate our method using the dataset from the 2015 PhysioNet Computing in Cardiology Challenge. Ablation analysis demonstrates that Contrastive Learning significantly improves the performance of a combined deep learning and rule-based-embedding approach. Our results indicate that the final proposed deep learning framework achieves superior performance in comparison to the winning entries of the Challenge.

Oriented Localization of Surgical Tools by Location Encoding.

Surgical tool localization is the foundation to a series of advanced surgical functions e.g. image guided surgical navigation. For precise scenarios like surgical tool localization, sophisticated tools and sensitive tissues can be quite close. This requires a higher localization accuracy than general object localization. And it is also meaningful to know the orientation of tools. To achieve these, this paper proposes a Compressive Sensing based Location Encoding scheme, which formulates the task of surgical tool localization in pixel space into a task of vector regression in encoding space. Furthermore with this scheme, the method is able to capture orientation of surgical tools rather than simply outputting horizontal bounding boxes. To prevent gradient vanishing, a novel back-propagation rule for sparse reconstruction is derived. The back-propagation rule is applicable to different implementations of sparse reconstruction and renders the entire network end-to-end trainable. Finally, the proposed approach gives more accurate bounding boxes as well as capturing the orientation of tools, and achieves state-of-the-art performance compared with 9 competitive both oriented and non-oriented localization methods on a mainstream surgical image dataset: m2cai16-tool-locations. A range of experiments support our claim that regression in CSLE space performs better than traditionally detecting bounding boxes in pixel space.

Sketch-Guided Scenery Image Outpainting.

The outpainting results produced by existing approaches are often too random to meet users' requirements. In this work, we take the image outpainting one step forward by allowing users to harvest personal custom outpainting results using sketches as the guidance. To this end, we propose an encoder-decoder based network to conduct sketch-guided outpainting, where two alignment modules are adopted to impose the generated content to be realistic and consistent with the provided sketches. First, we apply a holistic alignment module to make the synthesized part be similar to the real one from the global view. Second, we reversely produce the sketches from the synthesized part and encourage them be consistent with the ground-truth ones using a sketch alignment module. In this way, the learned generator will be imposed to pay more attention to fine details and be sensitive to the guiding sketches. To our knowledge, this work is the first attempt to explore the challenging yet meaningful conditional scenery image outpainting. We conduct extensive experiments on two collected benchmarks to qualitatively and quantitatively validate the effectiveness of our approach compared with the other state-of-the-art generative models.

Crowd Scene Analysis Encounters High Density and Scale Variation.

Crowd scene analysis receives growing attention due to its wide applications. Grasping the accurate crowd location is important for identifying high-risk regions. In this article, we propose a Compressed Sensing based Output Encoding (CSOE) scheme, which casts detecting pixel coordinates of small objects into a task of signal regression in encoding signal space. To prevent gradient vanishing, we derive our own sparse reconstruction backpropagation rule that is adaptive to distinct implementations of sparse reconstruction and makes the whole model end-to-end trainable. With the support of CSOE and the backpropagation rule, the proposed method shows more robustness to deep model training error, which is especially harmful to crowd counting and localization. The proposed method achieves state-of-the-art performance across four mainstream datasets, especially achieves excellent results in highly crowded scenes. A series of analysis and experiments support our claim that regression in CSOE space is better than traditionally detecting coordinates of small objects in pixel space for highly crowded scenes.